Reconstruction method of biological tissue image, apparatus therefor, and  image display apparatus using the biological tissue image

ABSTRACT

The present invention provides a method for classifying biological tissues with high precision compared to a conventional method. When measuring a spectrum which has a two-dimensional distribution that is correlated with a slice of a biological tissue, and acquiring a biological tissue image from the two-dimensional measured spectrum, the method includes dividing an image region into a plurality of small blocks, and then reconstructing the biological tissue image by using the measured spectrum and a classifier corresponding to each of the regions.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a reconstruction method of a biologicaltissue image and an apparatus therefor, and particularly relates to amethod for reconstructing a biological tissue image from measuredspectrum data which is correlated with a substance distributed within abiological tissue, and to an apparatus therefor. The present inventionalso relates to an image display apparatus for clearly displaying alesion at a pathological diagnosis by using thus acquired biologicaltissue image.

2. Description of the Related Art

Conventionally, a pathological diagnosis has been conducted which isspecifically a diagnosis for the presence or absence of a lesion and atype of the lesion, based on the observation for a biological tissue ofan object by a microscope or the like. In the pathological diagnosis, aconstitutive substance and a contained substance which are correlatedwith a biological tissue of an object to be observed are required to bevisualized. So far, a technique for staining a specific antigen proteinby using an immunostaining method has mainly been employed in thepathological diagnosis. When breast cancer is taken as an example, ER(estrogen receptor which is expressed in hormone-dependent tumor) whichserves as a determination criterion for hormone therapy and HER2(membrane protein to be found in fast-growing cancer) which serves as adetermination criterion for Herceptin administration are visualized bythe immunostaining method. However, the immunostaining method has suchproblems that the reproducibility is poor because an antibody isunstable and antigen-antibody reaction efficiency is difficult to becontrolled. In addition, when needs of such a functional diagnosis willbe grown in the future, for instance, and when there arises a need ofdetecting several tens or more types of constitutive substances orcontained substances, the currently-employed immunostaining method has aproblem of being incapable of meeting the need any more.

In addition, in some cases, the visualization of the substance which isdistributed within a biological tissue, such as the constitutivesubstance and the contained substance, is not sufficient at a tissuelevel, and the visualization at a cellular level is required. Forinstance, in research on a cancer stem cell, it was revealed that atumor was formed in only part of fractions of a tumor tissue afterxenotransplantation to immunocompromised mice, and accordingly, it isbeing understood that the growth of a tumor tissue is dependent ondifferentiation and self-reproduction abilities of the cancer stemcells. In such examination, it is necessary not to observe the entiretissue, but to observe an expression distribution of a constitutivesubstance or a contained substance in each of individual cells in atissue.

Incidentally, the above described “cellular level” means a level atwhich at least each of the individual cells can be classified. Adiameter of the cell exists in a range of approximately 10 μm to 20 μm(provided that large cell such as nerve cell has diameter of about 50μm). Accordingly, in order to acquire a two-dimensional distributionimage at a cellular level, the spatial resolution needs to be 10 μm orless, can be 5 μm or less, further can be 2 μm or less, and stillfurther can be 1 μm or less. The spatial resolution can be determinedfrom a result of, for instance, a linear analysis of a knife-edgespecimen. In other words, the spatial resolution is determined based onthe general definition of “a distance between two points at which signalintensities originating in a concerned substance in the vicinity of theboundary of a specimen are 20% and 80%, respectively.”

As described above, in the pathological diagnosis, the constitutivesubstance and the contained substance which are correlated with a lesionor a pathological tissue are required to be exhaustively visualized at acellular level. The lesion or the pathological tissue means, forinstance, a tumor tissue and the like. Candidates for a method of suchvisualization include secondary-ion mass spectrometry (SIMS) includingtime-of-flight secondary-ion mass spectrometry (TOF-SIMS). A massspectrum is used as a measured spectrum. Furthermore, the candidatesinclude also Raman spectroscopy. Usable measured spectra include spectrain an ultraviolet region, a visible region and an infrared region. Thesemeasurement methods can provide information at each of plural points ina space at high spatial resolution. Specifically, the measurementmethods can provide spatial distribution information on each peak valueof the measured spectrum which is correlated with a substance that is anobject to be measured, and accordingly, can determine a spatialdistribution of the substance in a biological tissue which is correlatedwith the measured spectrum.

An SIMS method is a method of obtaining a mass spectrum at each point ona specimen by irradiating the specimen with a primary ion beam anddetecting secondary ions which have been separated from the specimen. Ina TOF-SIMS, for instance, it is possible to obtain the mass spectrum ateach point on the specimen by identifying the secondary ion with the useof such a fact that a flight time of the secondary ion depends on a massm and an electric charge z of the ion.

A Raman spectroscopy acquires a Raman spectrum by irradiating asubstance with a laser beam which is a monochromatic light as a lightsource, and detecting the generated Raman scattering light with aspectroscope or an interferometer. A difference (Raman shift) between afrequency of the Raman scattering light and a frequency of incidentlight takes a value peculiar to the structure of the substance, andaccordingly the Raman spectroscopy can acquire the Raman spectrumpeculiar to an object to be measured.

In order to acquire biological information from data of the measuredspectrum, a conventional method has generated a classifier beforehand bymachine learning, and has applied the generated classifier to the dataof the measured spectrum of the specimen (see Japanese PatentApplication Laid-Open No. 2010-71953). On the other hand, it has beenattempted to overlap a measured spectrum image (spectrum information)with an optical image (morphological information) and display theoverlapped image, because a biological tissue image is indispensable ina pathological diagnosis (see Japanese Patent Application Laid-Open No.2010-85219). Incidentally, the machine learning described here means atechnique of empirically learning data which have been previouslyacquired, and interpreting newly acquired data based on the learningresults. Further, the classifier refers to determination criterioninformation to be generated by empirically learning a relationshipbetween previously acquired data and biological information.

Conventionally, an example of diagnosing a disease by applying theclassifier which has been generated by the machine learning is describedalso in Patent Document 1. The object to be diagnosed is one measuredspectrum data (for one point on space or whole specimen), and it has notbeen assumed to acquire the biological tissue image from a spatialdistribution of the measured spectrum. In addition, there is an exampleof overlapping the measured spectrum image (spectrum information) withthe optical image (morphological information), but there has been noexample of acquiring the biological tissue image by applying the machinelearning (classifier) to both the spectrum information and themorphological information. Specifically, such a method has not beendisclosed as to reconstruct a biological tissue image with highprecision, which displays a diagnosis result related to a presence or anabsence of a cancer and the like, from a result of having measured aspectrum having the spatial distribution for the biological tissue of anobject.

In addition, when the measured spectrum has the spatial distribution,the characteristics of the data are different between positions at whichthe data is measured, for instance, a datum measured in the middle partof the image is different in the characteristics from that measured inthe peripheral portion of the image. Accordingly, the classifiersuitable for the position needs to be applied according to the positionat which the data is measured. However, conventionally, such a methodhas not been disclosed as to have assumed such a situation.

SUMMARY OF THE INVENTION

According to one aspect of the present invention, there is provided areconstruction method of a biological tissue image using a signalprocessing apparatus based on a measured spectrum correlated with asubstance distributed within a biological tissue, which includes:acquiring, within an image region, the measured spectrum at each ofplural points within the biological tissue; dividing the image regioninto a plurality of small blocks; selecting one or more of peaks of themeasured spectrum in each of the small blocks; acquiring a classifiercorresponding to each of the small blocks; and acquiring the biologicaltissue image per each of the small blocks based on the correspondingclassifier.

According to another aspect of the present invention, there is provideda reconstruction method of a biological tissue image using a signalprocessing apparatus based on a measured spectrum correlated with asubstance distributed within a biological tissue, which includes:acquiring the measured spectrum of the biological tissue; acquiringmorphological information acquired from distribution information of apeak component in the measured spectrum; acquiring a classifier; andapplying the classifier to both of the measured spectrum of thebiological tissue and of the morphological information acquired from thedistribution information of the peak component in the measured spectrum,to acquire the biological tissue image.

The biological tissue image acquired by the method of the presentinvention can be used for pathological diagnosis and the like.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic view of an apparatus on which the presentinvention is mounted.

FIG. 2 is a schematic view of a spectrum signal having an intensitydistribution in a two-dimensional plane.

FIGS. 3A, 3B and 3C are conceptual views of peak components in aspectrum.

FIG. 4 is a flow chart of the present invention.

FIG. 5 is a flow chart of machine learning with the use of aclassification analysis of a block of the present invention.

FIGS. 6A, 6B and 6C are schematic views of the discriminant analysis ofan image block.

FIGS. 7A, 7B and 7C are views schematically illustrating a projectionaxis which maximizes a ratio of an inter-group dispersion to anintra-group dispersion.

FIGS. 8A, 8B and 8C are views schematically illustrating a state inwhich confounding occurs due to data with different conditions.

FIGS. 9A, 9B and 9C are schematic views illustrating a process ofdetermining a regression model by the discriminant analysis of theblock.

FIGS. 10A, 10B and 10C are views schematically illustrating a series ofprocesses of the present invention.

FIGS. 11A, 11B and 11C are views illustrating an application process ofa first exemplary embodiment of the present invention.

FIGS. 12A and 12B are views illustrating that classification conditionsare different among the blocks of the different images.

FIGS. 13A and 13B are views illustrating that confounding occurs indata.

FIGS. 14A and 14B are views illustrating an application result of theregression analysis in the first exemplary embodiment of the presentinvention.

FIGS. 15A and 15B are views illustrating an application effect of thefirst exemplary embodiment of the present invention.

FIGS. 16A and 16B are schematic views illustrating the case where aMahalanobis distance is relatively small and the case where theMahalanobis distance is relatively large.

FIG. 17 is a schematic view of an apparatus shown in a second exemplaryembodiment of the present invention.

FIGS. 18A and 18B are views illustrating a spectrum image and spectra,which have been used in the second exemplary embodiment of the presentinvention.

FIGS. 19A, 19B, 19C and 19D are views illustrating an effect of theselection of a feature value in the second exemplary embodiment of thepresent invention.

FIG. 20 is an image illustrating a result of the discriminant analysiswhich has been conducted in the second exemplary embodiment of thepresent invention.

FIG. 21 is an image illustrating a result of the present invention inthe second exemplary embodiment of the present invention.

FIG. 22 is a schematic diagram illustrating a concept of a higher-orderlocal autocorrelation (HLAC).

FIGS. 23A and 23B are images illustrating an effect of the applicationof multidimensional information in the second exemplary embodiment ofthe present invention.

DESCRIPTION OF THE EMBODIMENTS

Preferred embodiments of the present invention will now be described indetail in accordance with the accompanying drawings.

Embodiments of the present invention will be specifically describedbelow with reference to the flow charts and the drawings. Incidentally,the following specific example is one example of exemplary embodimentsaccording to the present invention, but the present invention is notlimited to any such specific embodiment. The present invention includesmeasuring a specimen having a composition distribution in a space, andcan be applied to results provided by any measuring method as long asthe measuring method can obtain the information of a measured spectrumcorrelated with a substance distributed within a biological tissue or apathological tissue contained in a lesion so that the informationcorresponds to positional information at each point and positions ofeach point in the space.

A view illustrated in FIG. 4 is a flow chart of image reconstructionaccording to the present invention. The embodiment will be describedbelow with reference to the drawing according to the order in this flowchart.

In the step of S101 in FIG. 4, a peak to be used in the imagereconstruction is selected. Here, the peak means a peak of signalintensity in the case of the measured spectrum (for instance, massspectrum) as illustrated in FIG. 3A. On the other hand, there is aspectroscopy which uses a spectrum in an ultraviolet region, a visibleregion and an infrared region, or a Raman spectroscopy which uses aRaman spectroscopic spectrum, as the measured spectrum. The spectrummeasured when such a spectroscopy has been used forms a measured signalillustrated in FIG. 3B. In this case, the signal intensity illustratedin FIG. 3C, which has been provided by the discretization of themeasured signal, forms peaks of the signal intensity. Next, in the stepof S102, the data is normalized and digitized. In the step of S103,multi-dimensional data is generated from the normalized and digitizeddata, which is formed of positions of each point at which the spectrumhas been measured in the space and of a spectrum (peak component)measured at each of the points in the space.

A view illustrated in FIG. 2 is a schematic view illustrating theintensity distribution of the measured spectrum which has been measuredin each of the points on the space. For instance, when a two-dimensionalplane is considered as a space in which signals are acquired, theinformation becomes three-dimensional data. Each of the points of thethree-dimensional space from which these three-dimensional data aregenerated is expressed by a coordinate (X, Y, Z). The components X and Yare coordinates on the two-dimensional space (XY plane) in which themeasured spectrum signal is contained, and correspond to FIG. 2A. Thecomponent Z is a measured spectrum signal at each of the points on theXY plane, and corresponds to FIG. 2B. Accordingly, the components X andY store the X-coordinate and the Y-coordinate of the point at which thesignal has been measured, and the component Z stores a value of themeasured signal corresponding to the intensity of each peak component.

In the step of S104 in FIG. 4, the signal is classified by the generatedclassifier, and an image is output. Machine learning, for instance, canbe used for the generation of this classifier. In this machine learning,a determination criterion is generated which connects the measured dataand the information on the biological tissue, from already acquired data(which is referred to as training data).

A view illustrated in FIG. 5 is a flow chart for generating theclassifier. The content will be described below with reference to thedrawing according to the order in this flow chart.

In the step of S201 in FIG. 5, a peak to be used in the imagereconstruction is selected. Next, in the step of S202, the image dataare divided into blocks. Here, the division into blocks means that animage region is divided into each of a plurality of small blocks. In thestep of S203, the classifier is generated in each block from the data ofeach of divided blocks, for instance, by machine learning. As for atechnique of the machine learning, such methods can be used as aFisher's linear discriminant method, a SVM (Support Vector Machine), adecision tree, and a random forest method which considers the ensembleaverage thereof. In the step of S204, a classification model which canbe applied to all of the image regions is generated by a regressionanalysis of classification conditions obtained in each of the imageblocks. Incidentally, this step may be omitted, and it is acceptable,instead, to reconstruct the biological tissue image per every imageblock by using the classifier generated in each image block, and thenintegrate these images by interpolation processing and the like togenerate the biological tissue image in the image region. The case willbe described below where the Fisher's linear discriminant method hasbeen employed, as one example of supervised machine learning.Incidentally, the discriminant analysis means the Fisher's lineardiscriminant method, and the classification conditions mean discriminantconditions which have been acquired by the application of the Fisher'slinear discriminant method.

The image region of an object may be all of the image regions to beacquired, and may also be an image region which has been partiallyselected. When the image region which has been partially selected is theobject, it is acceptable, for instance, to previously set the imageregion which is not the object such as an outer peripheral portion, inall of the acquired image regions, and set the image region except forthe previously set image region.

FIGS. 6A to 6C illustrate a process of separating and classifying aplurality of groups from spectrum data by the discriminant analysis. Awhite frame in FIG. 6A shows a region in which the spectrum data to beused as training data is acquired. FIG. 6B is a schematic view of thespectrum data to be used. Each spectrum of an object to be learned isaccompanied by a classification number (label) of the biological tissue,such as 1 for a cancer tissue and 0 for a normal tissue, for instance.FIG. 6C schematically illustrates such a state that a feature valuewhich has been acquired from the spectrum data is projected to a featurespace (classification space) and an optimal boundary line is determinedby the discriminant analysis. Here, the feature space means a space towhich the feature value is projected in order to classify the attributeof the data, and the feature value means a value suitable forclassification, which is generated from original data. A normalized peakintensity and the like can be considered as the feature value in thiscase.

FIGS. 7A to 7C schematically illustrate a state of inter-groupdispersion and intra-group dispersion which are projected components toa projection axis. FIG. 7B illustrates the inter-group dispersioncorresponding to a distance between gravity centers of each group, andthe inter-group dispersion is given by Expression (1).

{w ^(T)( x ₁ − x ₂)}²  [Expression 1]

In addition, FIG. 7C illustrates the intra-group dispersion equivalentto dispersion within each group, and the intra-group dispersion is givenby Expression (2).

$\begin{matrix}{{{\frac{1}{\text{?} - 2}\left\{ {{\left( {\text{?} - \text{?}} \right)\text{?}\text{?}w} + {\left( {\text{?} - 1} \right)\text{?}\text{?}w}} \right\}} = {\text{?}\text{?}}}\mspace{20mu} {S = {\frac{1}{\text{?} + \text{?} - 2}\left\{ {{\left( {\text{?} - 1} \right)\text{?}} + {\left( {\text{?} - 1} \right)\text{?}}} \right\}}}{\text{?}\text{indicates text missing or illegible when filed}}} & \left\lbrack {{Expression}\mspace{14mu} 2} \right\rbrack\end{matrix}$

The vector w in the above Expression means a coefficient vector shown inthe following Expression (3). The vectors x₁ and x₂ in the aboveExpression mean a sample average vector of each group shown by thefollowing Expression (4). The matrices S₁ and S₂ in the above Expressionmean a sample-variance covariance matrix of each group shown by thefollowing Expression (5). The expressions are expressions in the casewhere the feature space is two dimensional, respectively. In addition,n₁ and n₂ are the numbers of the data of each group.

$\begin{matrix}{\mspace{79mu} {w = \begin{pmatrix}w_{1} \\w_{2}\end{pmatrix}}} & \left\lbrack {{Expression}\mspace{14mu} 3} \right\rbrack \\{\mspace{79mu} {{\text{?} = \left( \text{?} \right)}{\text{?} = \left( \text{?} \right)}}} & \left\lbrack {{Expression}\mspace{14mu} 4} \right\rbrack \\{{\text{?} = \left( \text{?} \right)}\mspace{79mu} {{S\; 2} = \left( \text{?} \right)}{\text{?}\text{indicates text missing or illegible when filed}}} & \left\lbrack {{Expression}\mspace{14mu} 5} \right\rbrack\end{matrix}$

The Fisher's linear discriminant method is a method of determining anaxis that maximizes a ratio of the inter-group dispersion and theintra-group dispersion which are the projected components to the axis,and such an axis is given by Expression (6). In Expression (6), xrepresents a coordinate in a feature space, and a position at which areference numeral of H(x) changes becomes a boundary that distinguishesboth of the groups.

$\begin{matrix}{\mspace{79mu} {{{h(x)} = {{\left( {\text{?} - \text{?}} \right)^{T}S^{- 1}x} - {\frac{1}{2}\left( {\text{?} - \text{?}} \right)^{T}{S^{- 1}\left( {\text{?} + \text{?}} \right)}}}}{\text{?}\text{indicates text missing or illegible when filed}}}} & \left\lbrack {{Expression}\mspace{14mu} 6} \right\rbrack\end{matrix}$

FIG. 7A schematically illustrates a classification axis which isdetermined by the discriminant analysis.

FIGS. 8A to 8C schematically illustrate such a state that confoundingoccurs when the discriminant analysis is conducted with the use of datain a plurality of different image blocks. The confounding means such aphenomenon that the data are mixed when the data having differentproperties are used. The white frame in FIG. 8A shows the block in theimage of which the data is used. FIG. 8B schematically illustratesspectrum data corresponding to the blocks, and FIG. 8C schematicallyillustrates such a state that the data are mixed due to confoundingoccurring when those data are projected to the feature space.

FIGS. 9A to 9C schematically illustrate such a state that theclassification conditions (which are determined from Expression (6)) foreach image block are acquired by local management of the data, theacquired classification conditions are subjected to the regressionanalysis, and thereby classification models capable of being applied tothe image regions are acquired. Here, the local management of the datameans that the data is divided in such a degree that confounding of thedata does not occur. The white frame in FIG. 9A shows the block in theimage of which the data is used. FIG. 9B illustrates such a state thatthe discriminant analysis is applied to each of the image blocks. FIG.9C illustrates such a state that the regression analysis of theclassification conditions is conducted. Thus, the image is divided intoan appropriate image block size in such a degree that confounding doesnot occur. The classification model is constructed which can beappropriately applied to the image region, by the regression analysis ofthe classification conditions that have been acquired from thediscriminant analysis of each of the blocks. Thereby, it is enabled toconduct an appropriate classification while preventing the confoundingof the data. For information, the optimal image block size can bedetermined, for instance, by using such a statistical test as is givenby Expression (7), a misclassification rate of the training data, andthe like.

$\begin{matrix}{\mspace{79mu} {{\text{?} = \frac{\text{?} - \text{?}}{\sqrt{\frac{\text{?}}{\text{?}} + \frac{\text{?}}{\text{?}}}}}{\text{?}\text{indicates text missing or illegible when filed}}}} & \left\lbrack {{Expression}\mspace{14mu} 7} \right\rbrack\end{matrix}$

Here, σ₁ and σ₂ in Expression (7) mean a sample variance of each group.In addition, z₀ is a test value, and the block size is determined sothat the value becomes a constant value or more (for instance, 1.96 ormore).

FIGS. 10A to 10C schematically illustrate a series of processesillustrated in the flow charts in FIG. 4 and FIG. 5. In FIG. 10A, theclassification model is generated by the machine learning and theregression analysis, and in FIG. 10B, data which have been newlymeasured are input. Then, in FIG. 10C, a distribution image (which isobtained from result of machine learning) of the biological tissuedistribution, for instance, is acquired as a reconstruction image.

In addition, the data to be used in the machine learning and theclassification may not only be spectrum data of each point in the space,but also both the spectrum data of each point in the space and thedistribution information (morphological information) of each spectrumcomponent, for instance, may be used.

In this case, a peripheral area of a pixel which receives attention, forinstance, is cut out, and attention is paid to a pattern which theregion forms. For instance, when the two-dimensional plane is consideredas a space of which the signal is acquired, the data to be used in themachine learning and the classification shall be data having athree-dimensional structure in a total of the distribution informationand the spectrum information in the plane (which is referred to asmulti-dimensional information).

The procedure of the machine learning and the classification in the casewhere the multi-dimensional information has been used is essentially thesame as that in the case where the above described spectrum data hasbeen used. However, in this case, the data itself is not used for avector (which is referred to as feature vector hereafter) for use in theclassification, but also it is possible to acquire a plurality offeature values suitable for describing the pattern, define the featurevalues as a feature vector, and use the feature vector for the machinelearning and the classification processing. As a representative exampleof the feature value, there are a volume, a curvature, a space gradient,HLAC (high-order local autocorrelation) and the like. Here, thehigh-order autocorrelation function of N-order is defined by Expression(8) for displacement directions (a₁, a₂, . . . , a_(N)), when the imageof an object is represented by f(r).

x _(N)(a ₁ , a ₂ , . . . , a _(N))=∫f(r)f(r+a ₁) . . . f(r+a _(N))dr

In addition, the high-order local autocorrelation function is defined sothat the displacement directions are limited to a localized area of areference point r (for instance, 3×3 pixels around reference point r).FIG. 22 illustrates a reference pattern in the case of 0-order and1-order. A pixel with a charcoal gray becomes a center point for areference when the autocorrelation is calculated.

In addition, it is also possible to select the feature value to be usedin the machine learning beforehand. In this case, for instance, it isacceptable to calculate a Mahalanobis distance which is obtained byprojecting each of the feature values to the feature space and isdefined by the ratio of the inter-group dispersion and the intra-groupdispersion of each group, and to select the feature value for use in theclassification. The result that the Mahalanobis distance is smallcorresponds to the case as in FIG. 16A, and the result that theMahalanobis distance is large corresponds to the case as in FIG. 16B,when illustrated by Comparative Examples. If the Mahalanobis distance islarge, the classification becomes easier. Accordingly, it is alsopossible to preferentially select such a feature value that theMahalanobis distance between each group that receives attention islarge.

The present invention can be achieved by an apparatus which carries outthe above described specific embodiment. FIG. 1 illustrates one exampleof the configuration of the whole apparatus on which the presentinvention is mounted. A specimen 1 on a substrate and a detector 2 for asignal are shown. In addition, a signal processing apparatus 3 whichconducts the above described processing for the acquired signal, and animage display apparatus 4 which displays the signal processing result ona screen are shown.

The configuration will be more specifically described while taking aTOF-SIMS as an example. In the configuration, the detector 2 measuressecondary ions (which are shown by dotted line in FIG. 1) which aregenerated in the specimen 1 that has been irradiated with primary ions(not-shown), and transmits the signal which has been converted into anelectrical signal, to the signal processing apparatus 3. Forinformation, the type of the primary ion is not limited, and a usabledetector includes not only a detector for one dimension but also asemiconductor detector for two dimension. Furthermore, it is possible touse a laser in place of the primary ion, and also to use a specimenstage having a scanning function together. The measured data has athree-dimensional data structure in which a mass spectrum is stored in acoordinate point on the XY plane of the specimen 1. In addition, whenthe data has been integrated, the measured data becomes four-dimensionaldata. However, the integrated data becomes three dimensional, and can besubjected to similar processing.

In addition, FIG. 17 also illustrates one example of the configurationof the apparatus on which the present invention is mounted. A lightsource 11 and an optical system 12 are shown. In addition, the specimen1 to be measured, a stage 14 on which the specimen is arranged, and thedetector 2 for a signal are shown. In addition, the signal processingapparatus 3 which subjects the acquired signal to the above describedprocessing, and the image display apparatus 4 which displays the signalprocessing result on the screen are shown.

In FIG. 17, a measurement system of a transmission type of arrangementis shown, but a reflection type of arrangement is also possible. Inaddition, ultraviolet rays, visible light, infrared rays and the likecan be used as a light source. The detector also includes a singledetector, a line-shaped detector and a two-dimensional detector, and thetype is not limited. Furthermore, such a method is also acceptable as tocombine an interferometer with the apparatus and acquire a spectrumthrough Fourier transformation or Laplace transformation. It is alsopossible to add the scanning function to the specimen stage. Themeasured data has a three-dimensional data structure in which thespectrum is stored in the coordinate point on the XY plane of thespecimen 1. In addition, when the data has been integrated, the measureddata becomes four-dimensional data. However, the integrated data becomesthree dimensional, and can be subjected to similar processing.

In addition, FIG. 17 includes also nonlinear spectroscopy such ascoherent anti-Stokes Raman scattering (CARS, Coherent Anti-Stokes RamanScattering) and stimulated Raman scattering (SRS, Stimulated RamanScattering).

Furthermore, the signal processing apparatus and the image outputapparatus (that handle signal after detector) which are features of thepresent invention can be also applied to the configuration, as long asthe apparatus has such a structure that the spectrum is stored in thecoordinate point on one particular cross section (which has constantthickness) of the specimen. Specifically, the apparatuses can be appliedalso to a two-dimensional spectrum measuring system with the use ofX-rays, a terahertz wave, an electromagnetic wave or the like.

Exemplary Embodiment 1

Exemplary Embodiment 1 of the present invention will be described below.In the present exemplary embodiment, a tissue slice (product made byPantomics, Inc.) of an expression level 2+ of a HER2 protein which hadbeen subjected to trypsin digestion processing was subjected to an SIMSmeasurement on the following conditions, in which a TOF-SIMS 5 typeapparatus (trade name) made by ION-TOF GmbH was used.

Primary ion: 25 kV Bi⁺, 0.6 pA (pulse current value), macro-raster scanmode

Pulse frequency of primary ion: 5 kHz (200 μs/shot)

Pulse width of primary ion: approximately 0.8 ns

Beam diameter of primary ion: approximately 0.8 μm

Measurement range: 4 mm×4 mm

Pixel number in measurement of secondary ion: 256×256

Integration period of time: 512 shots for one pixel, one time scan(approximately 150 minutes)

Detection mode for secondary ion: positive ion

The XY coordinate information which shows the positions for eachmeasurement pixel and the mass spectrum in one shot are recorded in theobtained SIMS data. For instance, each of the measurement pixelscontains the information on the peak (m/z=720.35) which corresponds to amass number of a molecule in one of digestive fragments of the HER2protein, to which one sodium atom adsorbs, and on the peak componentsoriginating in each biological tissue, as the spectrum data.

FIG. 11A illustrates an image obtained through the observation of thetissue slice (product made by Pantomics, Inc.) of the expression level2+ of the HER2 protein, of which the HER2 protein was subjected toimmunostaining, by an optical microscope. In FIG. 11A, the portion inwhich there are more expressions in the HER2 protein is indicatedwhiter. In addition, the specimen that was used in the SIMS measurementand the specimen that was subjected to the immunostaining are adjacentslices to each other, which were cut out from the same diseased tissue(paraffin block), and are not identical.

FIG. 11B illustrates a spectrum which was measured in an image blocksurrounded by the white frame in FIG. 11A. FIG. 11C illustrates a resultof the discriminant analysis that was conducted for two peak components(values of corresponding m/z are 692.35 and 1101.5, respectively), whichwere selected from the spectrum of FIG. 11B. It is understood in FIG.11C that the different groups can be clearly separated from each other.

The white frames in FIG. 12A illustrate a plurality of image blocks.FIG. 12B illustrates a result of the discriminant analysis which wasconducted for the spectrum data of each of the image blocks. It isunderstood in FIG. 12B that the characteristics of the data changeaccording to the positions of the image blocks, and the classificationcondition also changes according to the change of the characteristics.

The white frame in FIG. 13A illustrates an image block formed by acombination of the plurality of the image blocks in FIG. 12A. FIG. 13Billustrates a result of the plotting of the data in the white frame inFIG. 13A on the feature space. It is understood that such a phenomenonthat data in different groups are mixed with each other, which isso-called confounding, occurs in the white frame in FIG. 13B.

FIG. 14A illustrates a result of the division of the image region intoimage blocks. FIG. 14B illustrates a result of the regression analysiswhich was conducted for the classification conditions based on theresult of the discriminant analysis that was conducted for a pluralityof the image blocks. From the result of this regression analysis, aclassification model is generated which can be applied to the imageregion.

FIG. 15A illustrates a result of the reconstruction of the image, by theapplication of the classification conditions obtained from thediscriminant analysis of a single image block, to the image region. Inaddition, FIG. 15B illustrates a result of the reconstruction of theimage, by the application of the discriminant analysis of the dividedblocks according to the present invention, to the image region. As isunderstood when the inside of the white frame is referred and comparedto FIG. 11A of reference, a biological tissue image with higherprecision can be acquired according to the present invention.

Exemplary Embodiment 2

Exemplary Embodiment 2 of the present invention will be described below.In the following exemplary embodiment, a mouse liver tissue was measuredwith the use of a microscope which uses stimulated Raman scattering. Thepower of a TiS laser used as a light source was 111 mW, and theintensity of an Yb fiber laser was 127 mW before the laser was incidenton an object lens. The mouse liver tissue of the specimen was subjectedto formalin fixation treatment, and was cut into a thin slice with athickness of 100 micrometers. The tissue slice was subjected tomeasurement in a state of being embedded in a glass together with a PBSbuffer. The measurement range is a 160 micrometers square, and the datameasured 10 times were integrated. The image data was a 500 pixelssquare, and the measurement period of time was 30 seconds.

The XY coordinate information which shows the positions of eachmeasurement pixel, and the spectrum information in each coordinate arerecorded in the obtained spectral image data. For instance, each of themeasurement pixels contains the information on the peak componentsoriginating in the components of the tissue constituting the specimen,as the spectrum data. In addition, the measurement of the spectrum datawas conducted with a sampling interval of 1 kayser (1 cm⁻¹).

FIG. 18A is an image formed by adding up of signals in all measuredspectral regions, which are based on a measurement result of a livertissue. FIG. 18B is a graph obtained from picked-up spectra of portionscorresponding to the cell nucleus, the cell cytoplasm and theerythrocyte; and in the graph, a horizontal axis corresponds to a wavenumber (while the numerical value in the graph is an index fordistinguishing the wave number and the index will be referred tohereafter), and a vertical axis corresponds to signal intensity. It isunderstood that spectrum signals different among each tissue areobtained, as is illustrated in FIG. 18B.

FIG. 19A is a graph obtained from the calculation of a Mahalanobisdistance between the cell nucleus (group 1) and the cell cytoplasm(group 2) for each wave number. It is understood that the Mahalanobisdistance is large when the indices are values between 7 and 8. FIG. 19Bis a graph obtained by determining the spectrum intensitiescorresponding to the indices 7 and 8 as the feature values, and plottingone part of the training data onto the two-dimensional feature space. Itis understood that the group 1 and the group 2 can be clearlydistinguished. FIG. 19C is a graph obtained from the calculation of aMahalanobis distance between the cell cytoplasm (group 2) and theerythrocyte (group 3) for each wave number. It is understood that theMahalanobis distance is large when the indices are values between 15 and17. FIG. 19D is a graph obtained similarly by determining the spectrumintensities corresponding to the indices 15 and 16 as the featurevalues, and plotting one part of the training data onto thetwo-dimensional feature space. It is understood that the group 2 and thegroup 3 can be clearly distinguished. On the other hand, it isunderstood that the group 1 and the group 2 result in resisting beingdistinguished. In such a case, it is acceptable to use all of thefeature values which are suitable for classification among each of thegroups, and then project the resultant feature values to the featurespace. In this case, it is acceptable, for instance, to determine thespectrum intensities corresponding to the indices 7, 8 and so on, and15, 16 and so on as the feature values, project the resultant featurevalues to the multidimensional feature space, and classify each group.

FIG. 20 is a result of the discriminant analysis which has beenconducted for the plotting of spectrum intensities corresponding to theindex 8 and the index 15 on a two-dimensional space, with the use oftraining data that correspond to the cell nucleus (group 1), the cellcytoplasm (group 2) and the erythrocyte (group 3), respectively. It isunderstood that each group can be clearly separated from each other.

FIG. 21 is an image obtained by classifying the cell nucleus, the cellcytoplasm and the erythrocyte based on the result of the previouslydescribed discriminant analysis, and reconstructing the resultant image.It is understood that each tissue is appropriately classified and iscolor-coded. Thus, the present invention can classify a structure in abiological tissue without dyeing.

FIG. 23A illustrates a result of classification processing that has beenconducted with the use of only spectrum intensity as a feature value;and FIG. 23B illustrates a result of classification processing that hasbeen conducted with the use of the spectrum intensity and HLAC in0-order and 1-order, which is calculated from the distribution of thespectrum intensity, as a feature value. It is understood that an outlineof a structure in each tissue is more clearly drawn in FIG. 23B comparedto that in FIG. 23A. Thus, the outline of the structure in a biologicaltissue can be more clearly drawn by the utilization of suchmultidimensional information.

The present invention can be used as a tool which more effectivelysupports a pathological diagnosis.

The method according to the present invention can reconstruct abiological tissue image, by measuring a spatial distribution of ameasured spectrum, using both measured spectrum information thereof andmorphological information which is obtained from the distributioninformation of peak components and applying machine learning to theinformation. Furthermore, even in the case where characteristics of datachange and classification conditions change due to a difference amongmeasured positions of the measured spectrum and the like, at this time,the method can reconstruct a biological tissue image by employingappropriate classification conditions. Thereby, the biological tissuecan be classified with higher precision compared to a conventionalmethod, and accordingly the method is useful when being applied to thepathological diagnosis or the like.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Applications No.2013-000883, filed Jan. 8, 2013, No. 2013-163399, filed Aug. 6, 2013 andNo. 2013-251050, filed Dec. 4, 2013 which are hereby incorporated byreference herein in their entirety.

What is claimed is:
 1. A reconstruction method of a biological tissueimage using a signal processing apparatus based on a measured spectrumcorrelated with a substance distributed within a biological tissuecomprising: acquiring, within an image region, the measured spectrum ateach of plural points within the biological tissue; dividing the imageregion into a plurality of small blocks; selecting one or more of peaksof the measured spectrum in each of the small blocks; acquiring aclassifier corresponding to each of the small blocks; and acquiring thebiological tissue image per each of the small blocks based on theselected one or more of peaks and the corresponding classifier.
 2. Thereconstruction method of the biological tissue image according to claim1, wherein the classifier to be applied to an image region is generatedfrom classification conditions acquired per each of a plurality of smallblocks, by a regression analysis of the classification conditions, andthen the biological tissue image in the image region is generated byapplying the classifier to the measured spectrum.
 3. The reconstructionmethod of the biological tissue image according to claim 1, wherein thebiological tissue images acquired per each of a plurality of smallblocks are integrated to generate the biological tissue image in theimage region.
 4. The reconstruction method of the biological tissueimage according to claim 1, wherein the classifier is generated byapplying a training data to the measured spectrum.
 5. The reconstructionmethod of the biological tissue image according to claim 1, wherein, atthe selecting one or more of peaks of the measured spectrum, the peakfor use in the classification is determined based on Mahalanobisdistance defined by a ratio of an inter-group dispersion to anintra-group dispersion.
 6. The reconstruction method of the biologicaltissue image according to claim 1, wherein, the measured spectrum is anyof a spectrum in an ultraviolet region, a visible region and an infraredregion, a Raman spectroscopic spectrum, and a mass spectrum.
 7. Thereconstruction method of the biological tissue image according to claim1, wherein, the biological tissue is a pathological tissue.
 8. Abiological tissue image acquiring apparatus, wherein a biological tissueimage is reconstructed by the method according to claim
 1. 9. An imagedisplay apparatus, wherein, at a pathological diagnosis, a lesion isdisplayed by the biological tissue image acquiring apparatus accordingto claim
 7. 10. A reconstruction method of a biological tissue imageusing a signal processing apparatus based on a measured spectrumcorrelated with a substance distributed within a biological tissuecomprising: acquiring the measured spectrum within the biologicaltissue; acquiring morphological information from distributioninformation of a peak component in the measured spectrum; acquiring aclassifier; and applying the classifier to both of the measured spectrumof the biological tissue and of the morphological information acquiredfrom the distribution information of the peak component in the measuredspectrum, to acquire the biological tissue image.
 11. The reconstructionmethod of the biological tissue image according to claim 10, whereinwhen the classifier is generated and the biological tissue image isreconstructed, both of the measured spectrum and a higher-order localautocorrelation which is acquired from distribution information thereofare used.